"When Birds of a Feather Flock Together": Synesthetic Correspondences Modulate Audiovisual Integration in Non-Synesthetes

Synesthesia is a condition in which the stimulation of one sensory modality elicits an additional experience, often in a different (i.e., unstimulated) sense. Although only a small proportion of the population is synesthetic, there is growing evidence to suggest that neurocognitively-normal individuals also experience some form of synesthetic association between the stimuli presented to different sensory modalities (i.e., between auditory pitch and visual size, where lower frequency tones are associated with large objects and higher frequency tones with small objects). While previous research has highlighted crossmodal interactions between synesthetically corresponding dimensions, the possible role of synesthetic associations in multisensory integration has not been considered previously. Here we investigate the effects of synesthetic associations by presenting pairs of temporally asynchronous or spatially discrepant visual and auditory stimuli that were either synesthetically matched or mismatched. In a series of three psychophysical experiments, involving unspeeded two alternatives forced-choice discrimination tasks, participants reported the relative temporal order of presentation or the relative spatial locations of the two stimuli. The results showed that the reliability of non-synesthetic participants’ estimates of both audiovisual temporal asynchrony and spatial discrepancy were lower for pairs of synesthetically matched (i.e. high-pitched tones and small visual stimuli) as compared to synesthetically mismatched audiovisual (i.e. high-pitched tones and large visual stimuli) stimuli. Recent studies of multisensory integration have shown that the reduced reliability of perceptual estimates regarding intersensory conflicts constitutes the marker of a stronger coupling between the unisensory signals. These results therefore indicate a stronger coupling between synesthetically matched vs. mismatched stimuli and provide the first psychophysical evidence that synesthetic congruency can promote multisensory integration. Synesthetic crossmodal correspondences therefore appear to play a crucial (if unacknowledged) role in the multisensory integration of auditory and visual information